Multi-axis integration system and method

ABSTRACT

An image acquisition system and method employing multi-axis integration (MAI) may incorporate both optical axis integration (OAI) and time-delay integration (TDI) techniques. Disclosed MAI systems and methods may integrate image data in the z direction as the data are acquired, projecting the image data prior to deconvolution. Lateral translation of the image plane during the scan in the z direction may allow large areas to be imaged in a single scan sequence.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of copending U.S. patent applicationSer. No. 11/202,745, filed Aug. 12, 2005 and titled “Multi-AxisIntegration System and Method,” which is a continuation of U.S. patentapplication Ser. No. 10/389,269 (the '269 application), filed Mar. 13,2003 which issued as U.S. Pat. No. 7,283,253 and which applications areincorporated herein by reference and for all purposes; the '269application was based on and claimed the benefit of U.S. provisionalapplication Ser. No. 60/364,762, filed Mar. 13, 2002, entitled “METHODFOR MULTI-AXIS INTEGRATION (MAI) IMAGING OF THICK SPECIMENS,” and U.S.provisional application Ser. No. 60/431,692, filed Dec. 6, 2002,entitled “OPTICAL AXIS INTEGRATION SYSTEM AND METHOD.” The presentapplication is also related to abandoned U.S. nonprovisional applicationSer. No. 10/215,265, filed Aug. 6, 2002, entitled “TIME-DELAYINTEGRATION IMAGING OF BIOLOGICAL SPECIMENS.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Aspects of the present invention relate generally to image acquisitionsystems and image processing techniques, and more particularly to asystem and method of integrating image data obtained along an opticalaxis and in conjunction with lateral translation of an image plane.

2. Description of Related Art

A variety of autofocus techniques have been developed to enableautomated and semi-automated scanning of three-dimensional (3D) objectsusing short depth-of-field (DOF) optics. Such short DOF optical systemsare routinely used in microscopes and other inspection systems, at leastpartially because a short DOF can provide superior image quality andresolution within the limits of the DOF. By way of background, theresolution of an image is directly proportional to the numericalaperture (NA) of the optics; consequently, high resolution imagesgenerally require high NA values. As indicated in Table 1, however, DOFis inversely proportional to the NA; accordingly, a high NA (i.e., highresolution) necessarily generates a narrow DOF which, in turn, increasesblurring in the final image since portions of the imaged object extendabove and below the plane of focus, or image plane.

In that regard, out-of-focus images are usually not useful for imageanalysis and are typically a cause of failure in automated scanning andimage acquisition systems. Accordingly, autofocus features havetypically been regarded as essential components of many conventional,short DOF automated systems.

Autofocus is not a trivial technology, however, and as a consequence,incorporating autofocus functionality presents a variety of complicatedchallenges associated with design, implementation, integration, and usein conjunction with common imaging systems. In particular, automaticallydetermining a desired or optimal focal plane can be an impossible taskwhen an object to be viewed or imaged is larger in the axial dimensionthan the DOF of the optics (see Table 1). In such cases, the object tobe imaged generally has a plurality of focal planes, none of which maybe “correct” or optimal. Even when an autofocus feature is successful inascertaining a “best” focal plane, the time required to determine thatplane of best focus can be a major limitation with respect to the scanrate.

TABLE 1 DOF examples (at 540 nm wavelength) for numerical apertures (NA)from 0.10 to 1.40. Refractive Type NA Index DOF (μm) Low NA air 0.101.00 54.00 Med NA air 0.20 1.00 13.50 Med NA air 0.40 1.00 3.38 Med NAair 0.45 1.00 2.67 High NA air 0.75 1.00 0.96 High NA water 1.20 1.350.51 High NA oil 1.35 1.52 0.45 High NA oil 1.40 1.52 0.42

As is generally known in the art, autofocusing methods may be employedin several ways. For example, autofocus techniques are typically used tocollect images of two-dimensional (2D) objects mounted or disposed onnon-planar surfaces. Further, autofocus is also commonly used to collectimages of 3D objects; in fact, most objects of interest with respect tooperation of typical image acquisition systems are 3D rather than simply2D. In some situations, any 2D image of the object may be consideredsufficient. In other situations, however, an image captured from aparticular, predetermined, or otherwise specified focal plane isdesired. In either case, the 2D image obtained with an autofocustechnique is ordinarily expected to contain valuable morphologicaland/or quantitative information about the object, which is usually 3D,as noted above.

Additionally, autofocus may be employed to locate a starting point for aseries of images to be obtained along the optical axis (so called“optical sections”); using image data from sequential optical sections,an image processing system may ascertain more information from theresulting 3D image than is possible from analysis of the individualcorresponding 2D images. It will be appreciated, however, that eachoptical section generally contains information from neighboring sections(i.e., sections above and below the focal plane of a given section) dueto the DOF range for a given NA, as illustrated in Table 1.

Images obtained from an autofocused system are typically analyzed withalgorithms that are based on some form of intensity integration. Forexample, the pixel intensities within a particular region of interest(e.g., on a microscope slide or a microarray) can be summed or totaledto infer the quantity of a particular chemical within that region;additionally or alternatively, pixels can be counted to determine thetotal area of the region. At the most basic level, certain existingtests are designed to identify the simple presence or absence of aparticular chemical; these tests generally rely upon an intensityintegration and are configured to yield a binary result.

In any event, conventional systems generally rely upon the sequentialacquisition of a plurality of optical sections followed by acomputationally expensive deconvolution operation. In particular, thedeconvolution is unnecessarily inefficient, since each particularoptical section contains blurred image data from neighboring sections asnoted above.

BRIEF SUMMARY OF THE INVENTION

Embodiments of the present invention overcome the above-mentioned andvarious other shortcomings of conventional technology, providing animage acquisition system and method employing integration of image dataobtained along an optical axis in conjunction with synchronizedtranslation in a lateral direction. A system and method operative inaccordance with some embodiments, for example, may integrate (orproject) image data in the z direction as the data are acquired;significant savings in computational overhead may be achieved byprojecting the image data prior to deconvolution. Lateral translation ofthe image plane during the scan in the z direction may provideadditional efficiency, allowing large areas to be imaged in a singlescan sequence.

Generally, an image acquisition system and method employing multi-axisintegration (MAI) in accordance with the present disclosure mayincorporate both optical axis integration (OAI) and time-delayintegration (TDI) techniques.

In accordance with some exemplary embodiments, a method of acquiringdata comprises: scanning an object along an optical axis; simultaneouslyscanning the object along a lateral axis; acquiring image data of theobject during the scanning and the simultaneously scanning; andintegrating the image data concomitantly with the acquiring. Thescanning may comprise providing relative translation along the opticalaxis of the object and an image plane; similarly, the simultaneouslyscanning may comprise providing relative translation along the lateralaxis of the object and an image plane.

Systems and methods are disclosed wherein the acquiring comprisesutilizing a charge-coupled device. In some methods, the integratingcomprises utilizing an image processor, and may further comprisedeblurring the image data subsequent to the integrating, deconvolvingthe image data subsequent to the integrating, or both.

In accordance with some exemplary implementations, a method of acquiringdata may further comprise calculating a two-dimensional projection ofthe object from a projection of the image data and a projection of anoptical point-spread-function.

The scanning may further comprise selectively alternating a direction ofthe relative translation along the optical axis. Additionally, thesimultaneously scanning may comprise synchronizing the relativetranslation along the lateral axis with a rate associated with theacquiring.

As set forth in more detail below, a method of acquiring image data ofan object may comprise: performing an optical axis integration scan;simultaneously executing a time-delay integration scan sequence; andselectively repeating the performing and the executing.

In some embodiments, the performing comprises acquiring image data ofthe object at an image plane positioned along an optical axis; theperforming may further comprise providing relative translation along theoptical axis of the object and the image plane; as noted briefly above,a direction of the relative translation may be selectively alternated.

The executing generally comprises providing relative translation along alateral axis of the object and the image plane; in some implementations,the executing comprises synchronizing the relative translation along thelateral axis with a data acquisition rate associated with an imagingdevice.

As set forth above, systems and methods are disclosed wherein: theacquiring comprises utilizing a charge-coupled device; the performingfurther comprises integrating the image data concomitantly with theacquiring; the integrating comprises utilizing an image processor; orsome combination thereof.

Image acquisition systems and methods in accordance with the presentdisclosure may comprise deblurring the image data subsequent to theintegrating, deconvolving the image data subsequent to the integrating,or both. A method may further comprise calculating a two-dimensionalprojection of the object from a projection of the image data and aprojection of an optical point-spread-function.

The foregoing and other aspects of various embodiments of the presentinvention will be apparent through examination of the following detaileddescription thereof in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a simplified functional block diagram illustrating oneembodiment of an image acquisition system operative in accordance withthe present disclosure.

FIG. 1B is a simplified functional block diagram illustrating a portionof the image acquisition system depicted in FIG. 1A.

FIGS. 2A-2C are simplified diagrams illustrating the general operationof embodiments of image acquisition and image data processing methods.

FIGS. 3A-3C are simplified flow diagrams illustrating the generaloperation of the methods depicted in FIGS. 2A-2C.

FIGS. 4A and 4B are simplified flow diagrams respectively illustratingthe general operation of embodiments of analog and digital optical axisintegration.

FIG. 5 is an illustration of results of an optical axis integrationmethod.

FIG. 6 is a simplified diagram illustrating the general operation ofshifts and readout functionality for a full frame CCD camera.

FIG. 7 is a simplified diagram illustrating one embodiment of time-delayintegration synchronized with parallel shifts.

FIGS. 8A and 8B illustrate representations of images acquired using oneembodiment of time-delay integration.

FIG. 9 is a simplified illustration of one embodiment of a CCD detectormask enabling acquisition of multi-spectral image data.

FIG. 10 is a simplified plot illustrating one embodiment of a multi-axisintegration scan cycle.

FIG. 11 is a simplified side view of an object to be imaged.

FIGS. 12A-12C are simplified side view illustrations of an image planetraversing an object to be imaged using one digital multi-axisintegration method.

FIG. 13 is an multi-axis integration summary.

DETAILED DESCRIPTION OF THE INVENTION

As noted briefly above, image acquisition throughput often representsthe rate-limiting factor in systems and methods of scanning high-contentand high-throughput assays common in biomedical and other applications.Image acquisition throughput can be especially problematic when an assayrequires detection of fluorescent probes, for example, and when highlateral resolution (in the x and y dimensions) is required forhigh-content image analysis algorithms. In cases where the detectedsignal is weak such as in fluorescence imaging for example, highnumerical aperture (NA) lenses are generally used to maximize collectionefficiency and to minimize exposure time. A side effect of high NAlenses, however, is that the depth-of-field (DOF, or the dimension ofthe in-focus region measured in the z direction) is very shallow. As setforth above, high NA lenses have limited ability to view thick objects,and are unable to follow uneven substrates without refocus.

Even in cases where the detected signal is strong or is otherwise easilyacquired (such as transmitted visible light, for example) opticalsystems can still perform inadequately if the sample thickness isgreater than can be imaged by the optical DOF; additional imagingdifficulties can be introduced if the object to be imaged is not locatedin a plane orthogonal to the optical axis. These optical limitationsoften lead to the use of autofocus technology, or the need to acquireimages at more than one focal plane.

Although much effort has been invested in autofocus technologies,optical axis integration techniques are more cost effective andgenerally provide improved performance in many scanning applications.The scanning techniques set forth in detail below are very tolerant ofobjects having inconsistent or variable focal planes, for example, andmay be used to image thick objects. Additionally, scans performed inaccordance with the present disclosure may be faster than thoseimplementing autofocus or optical sectioning procedures.

Optical Axis Integration

As an alternative to conventional autofocus methodologies, a system andmethod operative in accordance with the present disclosure employoptical axis integration (OAI) techniques as set forth in detail below.For a particular object to be imaged, for instance, rather thanattempting to determine a particular focal plane for optics or animaging apparatus (i.e., precisely determining an appropriate or optimalz position of the image plane), the object may be scanned along theoptical axis while a detector, computer, or other computationalapparatus concomitantly integrates the acquired images or image data.The resulting image is an integral (i.e., projection) of the image ofthe three-dimensional (3D) object along the optical axis. That is, anOAI image may generally be expressed as follows:

$\begin{matrix}{{i^{\prime}\left( {x,y} \right)} = {\int_{- \infty}^{\infty}{{i\left( {x,y,z} \right)}{z}}}} & 1\end{matrix}$

where i′ is the two-dimensional (2D) projection of a 3D image, i, alongthe optical axis (z direction).

In this context, the 3D image, i, can be described mathematically as theobject (o) of interest convolved with the point-spread-function (PSF) ofa microscope or other optical apparatus, as follows:

$\begin{matrix}{{i\left( {x,y,z} \right)} = {\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{{o\left( {x^{\prime},y^{\prime},z^{\prime}} \right)}{{psf}\left( {{x - x^{\prime}},{y - y^{\prime}},{z - z^{\prime}}} \right)}\ {x^{\prime}}\ {y^{\prime}}\ {z^{\prime}}}}}}} & 2\end{matrix}$

Inserting equation 2 into equation 1 gives

$\begin{matrix}{{i^{\prime}\left( {x,y} \right)} = {\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{{o\left( {x^{\prime},y^{\prime},z^{\prime}} \right)}{{psf}\left( {{x - x^{\prime}},{y - y^{\prime}},{z - z^{\prime}}} \right)}\ {x^{\prime}}\ {y^{\prime}}\ {z^{\prime}}\ {z}}}}}}} & 3\end{matrix}$

Rearranging the integration along the optical axis, z, then yields

$\begin{matrix}{{i^{\prime}\left( {x,y} \right)} = {\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{{o\left( {x^{\prime},y^{\prime},z^{\prime}} \right)}{\int_{- \infty}^{\infty}{{{psf}\left( {{x - x^{\prime}},{y - y^{\prime}},{z - z^{\prime}}} \right)}\ {z}\ {x^{\prime}}{y^{\prime}}{z^{\prime}}}}}}}}} & 4\end{matrix}$

which is equivalent to

$\begin{matrix}{{i^{\prime}\left( {x,y} \right)} = {\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{{o\left( {x^{\prime},y^{\prime},z^{\prime}} \right)}{\int_{- \infty}^{\infty}{{{psf}\left( {{x - x^{\prime}},{y - y^{\prime}},z} \right)}\ {z}\ {x^{\prime}}\ {y^{\prime}}\ {z^{\prime}}}}}}}}} & 5\end{matrix}$

Rearranging the integration along z′, the OAI image, i′(x,y), may beexpressed as:

$\begin{matrix}\begin{matrix}{{i^{\prime}\left( {x,y} \right)} = {\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{{o\left( {x^{\prime},y^{\prime},z^{\prime}} \right)}\ {z^{\prime}}}}}}} \\{\int_{- \infty}^{\infty}{{{psf}\left( {{x - x^{\prime}},{y - y^{\prime}},z} \right)}\ {z}\ {x^{\prime}}{y^{\prime}}}}\end{matrix} & 6\end{matrix}$

Equation 6 shows that an OAI image, i′(x,y), may be expressed as theconvolution of the integral of the object along the optical axis withthe integral of the PSF along the optical axis. Equation 6 is alsoillustrative of the relationship between the projection of the object,the projection of the image, and the projection of the PSF along theoptical axis.

The following definitions may facilitate further simplification of theforegoing formulation:

$\begin{matrix}{{{o^{\prime}\left( {x,y} \right)} = {\int_{- \infty}^{\infty}{{o\left( {x,y,z} \right)}\ {z}}}}{{{psf}^{\mspace{11mu} \prime}\left( {x,y} \right)} = {\int_{- \infty}^{\infty}{{{psf}\left( {x,y,z} \right)}{z}}}}} & 7\end{matrix}$

Inserting the definitions expressed above into Equation 6 yields

$\begin{matrix}{{i^{\prime}\left( {x,y} \right)} = {\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}{{o^{\prime}\left( {x,y} \right)}{{psf}^{\mspace{11mu} \prime}\left( {{x - x^{\prime}},{y - y^{\prime}}} \right)}\ {x^{\prime}}\ {y^{\prime}}}}}} & 8\end{matrix}$

The best method of solving Equation 8 for o′(x,y) involves FourierTransforms, and is a well known procedure. Applying a Fourier Transformto both sides of Equation 8 and applying the convolution theorem (see,e.g., Bracewell, 1986) results in the following relationship:

I′(u,v)=O′(u,v)OTF′(u,v)  9

Capital letters have been used to denote the Fourier Transform of thecorresponding functions, and the Fourier Transform of the PSF has beenreplaced with the conventional term for its Transform, the opticaltransfer function (OTF). Rearranging terms and performing an inverseFourier Transform then gives

$\begin{matrix}{{o^{\prime}\left( {x,y} \right)} = {F^{- 1}\left\lbrack {{I^{\prime}\left( {u,v} \right)}/{{OTF}^{\; \prime}\left( {u,v} \right)}} \right\rbrack}} & 10\end{matrix}$

where F⁻¹ represents the inverse Fourier Transform.

Equation 10 describes an efficient method of calculating a 2D projectionof an object from a projection of the image and a projection of theoptical PSF. A single-step solution may work well with good qualityimages; for lower quality images, however, an iterative solution ofEquation 10 may yield a more reliable result. See, e.g., the constrainediterative technique described by Agard et al. (David A. Agard and JohnW. Sedat, Nature, volume 302, 1984, pages 676 et seq.).

As described and contemplated in the present disclosure, a system andmethod may implement, incorporate, or comprise OAI techniques in eitherof two forms: digital; or analog. In embodiments incorporating orpracticing digital OAI, for example, a series of images may be collectedalong the optical axis and then digitally summed to form i′(x,y). Thissummation may occur during or after the scan, i.e., it may not benecessary to save individual optical sections as discrete images orcollections of image data. In analog OAI embodiments, for example,i′(x,y) may be generated by scanning the object along the optical axiswhile the image data are accumulated within a charge-coupled device(CCD) or other detector. The integration may be performed in the CCDchip and generally may result in only a single image, i.e., a singleimage may represent the entire depth of the object in the z directionalong the optical axis.

Analog OAI may have particular utility with respect to operationsinvolving scanning microarrays, for example, with CCD cameras or otherdetectors. A system and method employing analog OAI may eliminate orsubstantially reduce reliance upon sophisticated, time-consuming, andprocessor intensive autofocus procedures.

In many applications, analog OAI may provide a number of advantages overdigital OAI and autofocus, especially for automated scanners. Forexample, as compared with digital OAI, the analog OAI embodiments:require substantially lower data collection and processor overhead;exhibit lower read noise; and exhibit lower photon noise for equivalentexposure times.

As compared with traditional autofocus systems, advantages of the analogOAI embodiments may include the following: faster scan times; lowertotal exposure requirements; minimization or elimination of problemsrelated to determining an arbitrary plane of focus; and integration ofthe 3D object yields or allows full quantitation of the object, i.e.,the information content of the OAI image is higher than that achievedwith autofocus systems, and accordingly, fewer structures associatedwith the object of interest are missed.

As compared with the analog technology, advantages of digital OAIembodiments may include a potential for achieving substantially largerphoton counts; accordingly, 3D images may be made available for advancedimage analysis such as 3D deconvolution, volumetric measurements, andthe like.

The synergistic combination of the OAI techniques described above withdeconvolution, for example, may provide a significant advance forautomated slide scanning techniques. For instance, OAI images generallymay benefit from the quantitative deblurring procedure; similarly,deconvolution performance may be improved because Equation 10 deals withimages in 2D rather than 3D. Furthermore, many forms of image analysesbased upon images obtained from autofocused systems will work equallywell (or better) with projected images.

For example, a basic object detection operation may benefit from OAIimage processing techniques; in that regard, it will be appreciated thatimages with minimal DOF (i.e., autofocus images) are less likely tocontain a specific object of interest than the corresponding projectionimage. Likewise, analyses that use intensity integration may alsobenefit from application of OAI techniques, at least because the zdimension (i.e., along the optical axis) is already integrated into theOAI result. By way of another example, assays that integrate intensitieswithin 3D structures (e.g., nucleus, cytoplasm, and endoplasticreticulum) may generally be more accurate with OAI images because 2Dautofocus images cannot properly measure out-of-focus intensities.

Turning now to the drawing figures, FIG. 1A is a simplified functionalblock diagram illustrating one embodiment of an image acquisition systemoperative in accordance with the present disclosure, and FIG. 1B is asimplified functional block diagram illustrating a portion of the imageacquisition system depicted in FIG. 1A. Those of skill in the art willappreciate that FIGS. 1A and 1B are provided by way of example only, andthat the specific arrangement of components is susceptible of numerousmodifications; the exemplary scale, orientation, and interrelationshipof the various components may be altered in accordance with systemrequirements. Additionally, as will become apparent from examination ofthe following description, some or all of the functionality of somecomponents depicted as discrete elements may be combined or incorporatedinto other components.

System 100 generally comprises a microscope operably coupled to aprecision movable stage 120 and to an image acquisition component 140;stage 120 may be configured and operative to support a microarray,microscope slide, or other similar structure (reference numeral 190)upon which a specimen or object 199 to be imaged is disposed. As isgenerally known in the art, microscope 110 may comprise, or be operativein conjunction with, an illumination source 111 for illuminating stage120, slide 190, or both with light of a predetermined or selectedfrequency or spectral bandwidth; in that regard, illumination source 111may provide light in the visible, infrared, or ultraviolet wavelengths.

In some embodiments, illumination source 111 may be incorporated withinhousing 112 of microscope 110, i.e., on the opposite side of stage 120and slide 190 than depicted in FIG. 1A. Alternatively, an additionalsource of illumination (not shown) to be used in conjunction with, or inlieu of, source 111 may be accommodated or maintained in housing 112. Inthese embodiments, any such illumination source disposed within housing112 may be suitably dimensioned and positioned neither to interfere withoptical components of microscope 110 nor to obstruct the optical paththrough microscope 110 (to image acquisition component 140).

As noted above, stage 120 may be movable relative to optics (e.g.,objective 119 illustrated in FIG. 1B) incorporated into microscope 110(microscope optics are not depicted in FIG. 1A). In some embodiments,stage may be movable in both the x and y directions (where the y axis isnormal to the plane of FIGS. 1A and 1B). In this context, both the xaxis and the y axis may generally be referred to herein as “lateral”axes, and may describe a plane orthogonal to the optical axis (describedbelow) of system 100. Additionally or alternatively, stage 120 mayincorporate or comprise one or more structures and mechanisms configuredand operative precisely to position slide 190 laterally in the x and ydirections relative to the structure of stage 120 itself. In suchembodiments, precise 2D lateral positioning (i.e., x and y coordinates)of object 199 relative to the optical path of microscope 110 may beachieved through movement of stage 120 relative to microscope optics,movement of slide 190 relative to stage 120, or both.

In some embodiments, stage 120 may also be movable along the z axis (theoptical axis). It will be appreciated that microscope optics may alsofacilitate positioning an object on slide 190 in the proper location in3D space (i.e., x, y, and z coordinates) relative to the optical pathand the focal point of objective 119. In that regard, one or moreoptical components of microscope 110 such as objective 119 may bemovable in the z direction, either in addition to, or as an alternativeto, selectively moving stage 120 along the optical axis. Additionally oralternatively, objective 119 may be movable along the x axis, the yaxis, or both.

It will be appreciated that numerous mechanisms and methods ofpositioning object 199 to be imaged relative to microscope optics aregenerally known. Relative movement of various components (such as slide190, stage 120, and objective 119, for example), either individually orin combination, may vary in accordance with system requirements andconfiguration, and may be effectuated to position object 199 in asuitable location relative to objective 119. The present disclosure isnot intended to be limited by the structures and processes employed toposition object 199 relative to objective 119 and the optical path orthe image plane. Accordingly, reference made herein to relative motionof object 199 and an image plane may generally comprise movement ofobject 199, movement of the image plane, or some combination of both.

Microscope optics may generally be configured and operative inconjunction with image acquisition component 140; in that regard,component 140 generally comprises a camera, charge-coupled device (CCD),or other detector 141 operably coupled to an image processor 142 orother appropriate electronics. System 100 may additionally includecontrol electronics 150 operative to control, for example: operationalparameters, functional characteristics, or other configurable aspects ofimage processor 142 and detector 141; two- or three-dimensional motionof stage 120, objective 119, or other components; power output, spectralbandwidth, frequencies, or other parameters for source 111 and any otherillumination source incorporated into system 100; data storage; and thelike. In that regard, electronics 150 may comprise one or moremicroprocessors, microcontrollers, or other programmable devices capableof executing computer readable instructions; additionally, electronics150 may also comprise or be operably coupled with data storage media ornetworked devices such as file servers, application servers, and thelike. Those of skill in the art will appreciate that various methods andapparatus employing microprocessors or computer executable instructionsets to configure and to control operation of image acquisition systemsare generally known.

In operation, image data acquired by detector 141 may be summed,manipulated, saved, or otherwise processed by hardware, software, orboth resident at image processor 142; in some embodiments, functionalityof processor 142 may be influenced or controlled by signals transmittedfrom electronics 150 as noted above. Alternatively, the functionality ofimage processor 142 and electronics 150 may be incorporated into asingle device, for example. Specifically, image processor 142 may beoperative in accordance with instruction sets to compute solutions orapproximations for the equations set forth herein.

FIGS. 2A-2C are simplified diagrams illustrating the general operationof embodiments of image acquisition and image data processing methods,and FIGS. 3A-3C are simplified flow diagrams illustrating the generaloperation of the methods depicted in FIGS. 2A-2C.

FIGS. 2A and 3A generally illustrate one conventional approach to imageprocessing operations. As set forth above and indicated at block 311, aseries, or stack, of 2D images is acquired in sequential x,y planes(i.e., optical sections) along the z axis. The resulting image,i(x,y,z), is expressed mathematically at Equation 2 above, which iscomputationally expensive to solve. As illustrated in FIG. 2A, thedeconvolution operation depicted at block 312 is executed with respectto the entire stack of optical sections, and is accordingly inefficientand processor-intensive; since each optical section includes data fromother sections (due to DOF range, for example), the deconvolutionoperation processes more data than required. Finally, the deconvolved 3Dimage is projected into 2D image, o′(x,y), as indicated at block 313.

FIGS. 2B and 3B generally illustrate a significantly improved approachto image processing operations as contemplated herein. As in the FIG. 3Aembodiment, a series, or stack, of 2D images may be acquired insequential x,y planes (i.e., optical sections) along the z axis (block321). The resulting image, i(x,y,z), is expressed mathematically atEquation 2 above, which is computationally expensive to solve. Asillustrated in FIG. 2B and indicated at block 322, the stack of opticalsections may be projected into 2D image, i′(x,y), prior todeconvolution; this image is expressed mathematically at Equation 8above, which is a substantially simplified, 2D version of Equation 2.The deconvolution operation depicted at block 323 results in the same 2Dimage, o′(x,y), as the embodiment described above with reference toFIGS. 2A and 3A; the FIG. 3B embodiment generates the deconvolvedprojection at a significant savings in computational overhead, however,since the processor-intensive deconvolution is executed in only twodimensions.

FIGS. 2C and 3C illustrate one embodiment of an image acquisition andprocessing methodology employing OAI techniques for efficient productionof 2D images of 3D objects. In accordance with this embodiment, an OAIimage, i′(x,y), may be captured directly from the object (block 331),for example, by integrating x,y image data across a selected orpredetermined range along the optical axis in the z direction; in thatregard, relative translation of the object and the image plane may beprovided during the OAI scan as set forth above. The OAI image, i′(x,y),like the projected image illustrated in FIG. 2B, is expressed atEquation 8 above. In the exemplary embodiment of FIGS. 2C and 3C,however, the OAI image may be generated as the image data are obtained;the integration in this instance may be said to occur concomitantly orsubstantially simultaneously with the data acquisition. Accordingly,image data for a stack of discrete optical sections need not beacquired; this results in a significant reduction in data processingoverhead. Efficient deconvolution performance at block 332 may beachieved in part because the OAI image is expressed in 2D rather than in3D.

FIGS. 4A and 4B are simplified flow diagrams respectively illustratingthe general operation of embodiments of analog and digital optical axisintegration. It will be appreciated that the exemplary embodimentsillustrated in FIGS. 4A and 4B may generally correspond to theintegration operation depicted at block 331 and described above withreference to FIG. 3C; the illustrated OAI techniques may enablegeneration of an OAI image, i′(x,y), such as expressed at Equation 8.

In both the analog (FIG. 4A) and digital (FIG. 4B) OAI methods, a targetobject to be imaged may be identified and positioned (blocks 411, 421)relative to microscope optics substantially as set forth above; withrespect to positioning at blocks 411 and 421, only the x and ycoordinates of the object may be considered, i.e., the object may beaccurately aligned with the optical path. The operations depicted atblocks 412 and 422 are intended to represent any type of initiation orenablement of data capture or data acquisition; specifically, suchinitiation is not intended to be limited to physical movement ormanipulation of a camera shutter, for example, nor to imply otherstructural limitations. In that regard, the functionality represented atblocks 412 and 422 may include, inter alia, providing power to a CCD,for instance. Similarly, the operations depicted at blocks 415 and 424are not intended to imply structural limitations, and generallyencompass numerous techniques for disabling data capture functionality,for example, such as by closing a camera shutter.

In the analog embodiment of FIG. 4A, the object may be moved along theoptical axis through a predetermined or specified range (i.e., distancein the z direction) as indicated at block 413. It will be appreciatedthat the movement represented at block 413 may generally be measuredrelative to microscope optics or an image plane, for example, as setforth in detail above. Specifically: the object itself may remainstationary while the optics are moved; the optics may remain stationarywhile the object is moved; or some combination of system components maybe moved in cooperation.

As noted above, an analog method of OAI may comprise obtaining imagedata (block 414) continuously during the z axis scan (i.e., relativetranslation of the object and the image plane along the optical axis).Specifically, the OAI image may be generated by scanning the objectalong the optical axis while the image data are accumulated within a CCDor other detector 141. Accordingly, the integration or projection may beperformed during (concomitantly or substantially simultaneously with)data acquisition, and the image data written to memory at block 416generally may represent a single image, i′(x,y), such as depicted in thecenter of FIG. 2C. The method of FIG. 4A, therefore, may result in orfacilitate generation of a single 2D image representing the entire depthof the object in the z direction. As noted above, the method of FIG. 4Amay generally correspond to the operation depicted at block 331.

In contrast to the analog method, the digital OAI embodiment of FIG. 4Bmay obtain image data (block 423) in discrete steps. As illustrated bythe iterative loop containing blocks 422-424, acquired image datagenerally represent a plurality of 2D (x,y) sections; image datarepresenting discrete optical sections are written to memory (block 425)and summed or otherwise manipulated (block 426) as acquired at each zposition. In particular, the integration or projection may occur asimage data for each optical section are captured, i.e., concomitantlywith the data acquisition. Accordingly, generation of a 3D image,i(x,y,z), such as illustrated at the left side of FIGS. 2A and 2B, forexample, may not be necessary.

In accordance with a determination made at decision block 431, themethod may progress to the next optical section in sequence, moving theobject, the microscope optics, or both, so as to position the focalplane at the next sequential position along the z axis of the object;this event is represented at block 427. Following a progression througha desired, selected, or predetermined range in the z direction (asdetermined at decision block 431), the method may end (block 428),resulting in or allowing generation of a single, two-dimensional OAIimage, i′(x,y), representing the entire depth of the object in the zdirection. As noted above, the method of FIG. 4B may generallycorrespond to the operation depicted at block 331.

FIG. 5 is an illustration of results of an optical axis integrationmethod. It will be appreciated that the OAI images illustrated in FIG. 5may be enabled by the procedures set forth and described above withreference to FIGS. 4A and 4B.

Time-Delay Integration

As used herein, the phrase “time-delay integration” (TDI) generallyrepresents a method of continuous scanning which may be implemented inconjunction with CCD cameras or other imaging devices. In CCD cameras,for example, incident light is creates electric charge at individualcharge-coupled wells on the device surface. Charged electrons are thentransferred sequentially down the columns of the chip (parallel shifts)while the row that reaches the bottom of the chip is transferred to anaccumulator called the serial register. The serial register is thenshifted horizontally and processed by an A/D converter.

In accordance with some TDI embodiments, precision motion control may beemployed to synchronize motion of the object being imaged or motion ofthe camera or other imaging device (as set forth above with reference toFIG. 1A) with motion of the charged electrons across the CCD or imagingdevice surface. Relative translation of the object and the image planealong the lateral axis may be controlled such that a particular portionof the imaged object tracks down the chip as the electrons representingthat particular portion of the image are shifted down the chip. As setforth in detail above, such relative translation may comprise motion ofthe object, motion of the image plane, or both. Accordingly, the objectmay be continuously imaged as it passes down the chip. TDI methodologiesmay facilitate or enable efficient scanning of fluorescent DNAmicroarrays, for example, and may have utility in various otherapplications related to scanning myriad biological specimens.

FIG. 6 is a simplified diagram illustrating the general operation ofshifts and readout functionality for a full frame CCD camera. In thatregard, FIG. 6 provides a simple demonstration of slow-scan CCD cameraread operations. Individual pixel electrons are shifted in parallel(e.g., down the columns to successive rows) to a predetermined portionof the chip (e.g., the bottom of the chip in FIG. 6). Image data at thebottom row are shifted off of the chip onto the serial register, whichis, in turn, shifted horizontally to the readout amplifier to create avoltage that is digitized to form a digital image.

It will be appreciated that the FIG. 6 embodiment is provided forillustrative purposes only, and that various CCD cameras or otherimaging devices may be characterized by alternative operationalfeatures, particularly with respect to the exemplary geometry. Forexample, the operation of some CCD cameras may execute parallel shiftsoriented at 90 or 180 degrees from those depicted in FIG. 6.

FIG. 7 is a simplified diagram illustrating one embodiment of time-delayintegration synchronized with parallel shifts. In that regard, FIG. 7illustrates a precise motion control TDI implementation which may beemployed in fluorescence imaging systems, for example, or in numerousother imaging applications. In accordance with the exemplaryembodiments, a given location on the specimen (“object” in FIG. 7) maymove (either relative to the CCD chip, for example, or relative to theimage plane of the system) in synchrony with the parallel shifts suchthat VP (i.e., the parallel shift velocity) is equal to VY (i.e., theimage shift velocity). In the foregoing manner, the specimen may beimaged throughout the period of time that it takes for an entire chip tobe read by the camera.

In this context, synchronous motion between the object and the CCD rowmay be effectuated substantially as set forth in detail above withreference to FIG. 1A. Relative motion of slide 190, stage 120, variousoptical components, or some combination thereof, for instance, mayaccurately position the object depicted in FIG. 7 in a suitable locationfor imaging during the scan. The degree of synchronicity, the velocitiesimparted to the mechanical components, the precision with which theobject must be positioned for a desired imaging quality, mechanicalsettling time, backlash, and the like may vary in accordance with systemconfiguration, and may be application dependent.

Upon examination of FIGS. 6 and 7, it will be readily apparent thatrelative translation of the object and the image plane along the lateralaxis as set forth above may be synchronized with the rate at which dataare acquired and read from the detector. Accordingly, it is noted thatthe capabilities of the CCD camera or other imaging device (such asdetector 141) and other components of the optical system (such as, interalia: the maximum resolution and the time required to focus the imagingdevice; and NA and DOF of the optics) may affect the behavior andperformance of the system employing the FIG. 7 TDI embodiment.

FIGS. 8A and 8B illustrate representations of images acquired using oneembodiment of time-delay integration. In the TDI system used to generatethe exemplary images, the period of time for reading the entire CCD chipis approximately 0.4 seconds.

FIG. 8A is a representation of an image of a dirty blank slide scannedusing a TDI technique. Vertical bands indicate the locations ofindividual TDI “strips,” or portions of the imaged area. Followingcompletion of a particular TDI strip, the sample, the imaging device, orboth, may then be moved horizontally and a new TDI strip may then beacquired. Multiple strips may then be assembled to create the finalimage. FIG. 8B depicts a comparison of a region of a fluorescentmicroarray sample imaged first with individual panels and then with oneembodiment of a TDI method. It is noted that the individual panelborders are visible both horizontally and vertically in the “standardscan” image on the left side of FIG. 8B, while only the vertical bandsare visible in the TDI scan on the right. Both of these images wereacquired without calibrating the illumination and collectionefficiencies across the field (i.e., flat-field calibration).

Various embodiments of TDI may be employed to image objects inapplications involving limited or minimal signal intensity.Specifically, the motion control characteristics of TDI allow for longerexposure per picture element (pixel) for a given total image collectiontime, facilitating increased imaging quality even in instances wheresignal intensity ordinarily would be a deleterious factor. In manybiological sample imaging applications, for example, signal intensitymay be limited or otherwise impeded either by the nature of the sample,by the illumination intensity, by the physical characteristics oroperational parameters of the indicators used in conjunction with thesample, or by some combination of the foregoing.

By way of example, one application in which the above-mentioned factorsare especially problematic is in the imaging of fluorescently labeledbiological specimens. In imaging applications involving such samples,all three limitations noted above (ie., related to the sample, theillumination source, and the indicator employed) are prevalent.Accordingly, TDI methodologies may be used in conjunction with knownfluorescence imaging technology to minimize the attendant effects ofweak signal intensities.

While it will be appreciated that TDI techniques may prove useful in thecontext of multiple panel collection imaging schemes producing imagessuch as those illustrated in FIGS. 8A and 8B, some TDI systems andmethods may employ a single, wide detector; in that regard, a broaddetection apparatus may be used to collect a single “strip” thatrepresents the final image. Specifically, where detector 141, a CCDcamera, or other imaging apparatus accommodates a sufficiently broadimaging surface, for example, the multiple strips illustrated in FIGS.8A and 8B may be acquired in a single scan.

As mentioned above, one of the difficulties associated with scanningbiological specimens, especially for fluorescence characteristics, isgenerally due to the fact that a finite limit exists with respect to theintensity of light emanating from an illuminated sample. In accordancewith the present disclosure, however, adjusting the scan rate (i.e., therelative movement of the sample across the CCD imaging surface) and thereadout speed of the imaging device enables a system and method of TDIimaging to control the exposure time for each pixel in the acquiredimage.

Additionally, it will be appreciated that many samples (in biologicalfields and in other scientific areas) are labeled with multipleindicators, each of which may be spectrally separated the others.Consequently, some TDI embodiments may incorporate an ability to addressmultiple wavelength data. By way of example, data spanning multiplewavelengths may be acquired in at least two different ways: sequentialscanning; and simultaneous scanning.

In TDI implementations incorporating sequential scanning techniques, asingle, monochromatic detector may be employed; in this embodiment,multiple wavelengths may be selected through filters, for instance, orthrough other wavelength selection components or methods. To construct asingle, multiple wavelength image, an instrument or system operative inaccordance with the present disclosure may scan the sample (or a stripof the sample) through a selected or predetermined filter to acquireimage data at a desired wavelength, change the filter (using a filterwheel, for example, or similar apparatus), and scan the same sample (orstrip of sample) through the new filter to acquire additional image dataat a different desired wavelength. The foregoing operations may beselectively repeated in accordance with system requirements to acquire aplurality of scan images at a desired number of selected wavelengths. Inthis manner, a final image may be constructed of the data acquiredduring the plurality of sequential scans.

It will be appreciated that one of the challenges associated with such amethodology is the registration of scans, particularly in instanceswhere the exposure time used for one wavelength differs from theexposure time used for another. In such situations, a TDI system andmethod may measure and control the actual velocities and positions ofthe sample during each of the plurality of scans; precise control ofscan speeds, and accurate measurements thereof, may prevent or allow forcorrection of chromatic shift in the portions of the image that arederived from the component scans. Accordingly, systems and methods ofTDI as described herein may selectively synchronize the movements as setforth above responsive to such different conditions.

In some embodiments, multi-spectral image data may be acquired from asingle sample using multiple detectors, for example, each with its ownspectral response. In this manner, a multiple wavelength image may becollected in a single scan. Alternatively, such a multi-spectral scanmay be accomplished with a single detector or imaging device equippedwith a specially designed color mask. In that regard, FIG. 9 is asimplified illustration of one embodiment of a CCD detector maskenabling acquisition of multi-spectral image data. The mask may bearranged in alternating columns of red (R), green (G), and blue (B), forexample, as depicted in FIG. 9. Each of these respective columns in themask may facilitate the respective acquisition of red, green, and bluespectral image data. While the exemplary embodiment illustrates columns,it will be appreciated that the alternating pattern may be oriented insuch a way as to be parallel to the scan direction of the imagingdevice, i.e., where the scan occurs in rows, or 90 degrees from the scandirection illustrated in FIG. 7, the mask may be arranged to accommodatethe operational characteristics of the imaging device. As opposed to thesequential scan described above, a single, multi-wavelength,simultaneous scan may be implemented in conjunction with a mask such asillustrated in FIG. 9.

It is noted that positioning methodologies in various TDI embodimentsmay employ constant relative velocity of the object to be imaged and theimaging surface of the imaging device. As set forth above, constantrelative positioning of the object to be imaged may be accomplishedthrough precise motion of the slide, the stage, the optics, or somecombination thereof. In addition, it is possible to implement TDImethods employing one or more incremental positioning strategies; inthat regard, relative motion of the object to be imaged may besynchronized to the readout (or output) capabilities of the CCD cameraor other imaging device. In this manner, long exposures may beaccommodated.

In such embodiments, the velocity of the object may become so slow as tobecome non-constant, depending upon the output bandwidth and readoutrate of the imaging device. In some circumstances, for example, thesample or object may be translated (relatively) a distance correspondingto one camera row, maintained at that location for the duration of anexposure, and subsequently translated a distance corresponding to thenext camera row. The foregoing procedures may be selectively repeated inaccordance with system requirements or as desired. As set forth above,relative movement of the object, the slide, the stage, the optics, orsome combination thereof, may begin and cease at each row shift in thecamera or imaging device. The high degree of synchronization provided bysuch an approach may yield excellent resolution and registration betweenwavelengths.

Rotational and positional errors in precision motion stage systems maybe measured and corrected, for example, using optically determinedcoordinate calibration or other methods. In particular, such errors maybe minimized dynamically through precise modification of stagemovements. Specifically, such coordinate calibration may also be used inconjunction with. TDI techniques dynamically to correct rotational andpositioning errors during a given TDI scan.

Multi-Axis Integration

As noted generally above, a system and method employing multi-axisintegration (MAI) may incorporate both OAI and TDI techniques.

In that regard, TDI is described in co-pending U.S. nonprovisionalapplication Ser. No. 10/215,265, filed Aug. 6, 2002, entitled“TIME-DELAY INTEGRATION IMAGING OF BIOLOGICAL SPECIMENS.” Specifically,TDI methodologies as set forth in detail above enable a system andmethod of image data processing simultaneously to collect image dataover a given lateral area and to read data out of the camera ordetector. That is, data are acquired and read from the detectorsimultaneously. Accordingly, a system implementing TDI methods mayefficiently acquire image data over a large area (relative to the focalplane of the detector) with substantially reduced overhead as comparedto systems which do not implement TDI.

U.S. provisional application Ser. No. 60/431,692, filed Dec. 6, 2002,entitled “OPTICAL AXIS INTEGRATION SYSTEM AND METHOD” describes variousmethodologies for collecting image data as an integration (along theoptical axis) of intensity information for a given x-y image frame. Asset forth in detail above with reference to FIG. 4A, for example, someanalog OAI techniques utilize relative translation (along the opticalaxis) between the optical train and the specimen during integration toachieve an analog summing of the intensities over the depth of the ztranslation.

While the OAI methodologies set forth above are powerful, the disclosedembodiments may be augmented with one or more TDI scanning techniques.As set forth above with reference to FIGS. 4A and 4B, an OAI image maybe generated through integration of data acquired during a z translation(i.e., parallel with the optical axis as illustrated in FIG. 1B); imagedata may be read from the CCD camera or other detector 141 beforeanother integration begins. Depending upon various operationalcharacteristics of detector 141 and the capabilities of the associatedelectronics in processor 142, the readout may require up to one second,during which time many aspects of the scan must stop. This inefficiencydue to downtime may be avoided if the z translation is coordinated withthe lateral translation (i.e., motion in the x and y directions,orthogonal to the optical axis) attendant with the TDI scan as notedabove. That is, data gathered by detector 141 may be read continuouslyduring the z translation (resulting in an integration of intensitiesalong z) and simultaneously with lateral movement (such as along the yaxis) which presents new areas of the sample to detector 141; in someembodiments synchronizing such lateral movement with the readoutcapabilities of the CCD camera or other imaging device (i.e., detector141), additional computational efficiency may be achieved. During suchan MAI scan, the camera or detector 141 generally operates to acquireimage data in cooperation with continuous and simultaneous z- andy-motion.

In that regard, a system and method incorporating MAI techniques maygenerally acquire a plurality of OAI images, each of which may includedata captured from a different lateral location in x-y space on slide190 or stage 120. The lateral scan may employ a raster or serpentinepattern, for example; in some embodiments, a serpentine lateral scanpattern may enable an MAI system and method to operate at optimumefficiency.

Various data collection parameters may be optimized to maintainquantitative accuracy in accordance with system requirements. Inparticular, z-axis motion may be suitably controlled to assure anappropriate scan profile; accordingly, the resulting OAI image may notbe biased with information from any particular z location in the sampleor the object to be imaged. Additionally, the period of the scan profilemay be computed as a multiple of the time required to read a whole frameof data from detector 141 as set forth above in detail with reference toFIGS. 6 and 7.

FIG. 10 is a simplified plot illustrating one embodiment of a multi-axisintegration scan cycle. As indicated in FIG. 10, the z position of theimage plane along the optical axis is represented on the ordinate axis,and the y position of the optical axis relative to the stage 120 orslide 190 upon which the object to be imaged is disposed is representedon the abscissa. It will be appreciated that similar results may beobtained by providing lateral movement along the x axis, either inaddition to, or as an alternative to, motion along the y axis. As setforth in detail above with reference to FIG. 1A, accurate positioning ofthe focal plane relative to slide 190 and the area to be imaged may beeffectuated by relative movement of various components (such as slide190, stage 120, and objective 119, for example), either individually orin combination.

Various MAI techniques may incorporate either continuous (analog) orincremental (digital) scanning methodologies, as set forth below.

Analog MAI:

In accordance with some embodiments employing an analog MAI technique,the z location of the image plane may be moved continuously along theoptical axis between a minimum and maximum position (i.e., in and out offocus). The velocity of this z translation may be determined orcalculated as a function of the required scan distance (i.e., thedistance between the minimum and maximum z locations of the imageplane), as well as a desired scan time period. Acceleration may be madeas large as possible such that, for example, the position-versus-timecurve of the image plane may have a triangular wave pattern. A systemoperating in accordance with the analog MAI scan pattern illustrated inFIG. 10, for example, may facilitate uniform exposure at each z locationwithin the scan.

During translation of the image plane through the z scan, the y axis maybe simultaneously scanned at constant velocity such that the imagetraverses the entire detection surface of detector 141 while the ztranslation completes a cycle. Accordingly, each row of detector 141 maybe exposed to each z position four times as indicated by the pointsmarked z.sub.a in FIG. 10, i.e., every z position may be scanned fourtimes during a single MAI cycle. In such an embodiment, the exactstarting location or position (in the z dimension) of the object to beimaged may be irrelevant with respect to the quality and the efficiencyof the scan, since the object will cross each z position four timesduring a particular MAI cycle. It will be appreciated that for any givenlateral translation, higher z scan frequencies (i.e., greater than fourper MAI cycle) are also possible.

Digital MAI:

In accordance with digital MAI methodologies, the y and z axistranslations may be executed incrementally while rows of image data areread from the detector.

By way of example and not by way of limitation, the number ofincrements, n.sub.z, along the z axis in some embodiments of a digitalMAI cycle may be expressed as

n _(z)=3n _(y)−1

where n_(y) represents the number of lateral MAI increments in a singleMAI cycle. In the case of a CCD detector, for example, n_(y) may simplyrepresent the number of binned CCD rows:

n_(y)=number_binned_CCD_rows

In some implementations, the scan may also be constrained by thefollowing equation for n_(z):

n _(z)=(total scan range along z)/(row height)

When a CCD chip is being used as detector 141, the row height is thebinned CCD row height.

It will be appreciated that integral multiples of n_(z) may also be usedfor certain scanning techniques. Further, it will also be appreciatedthat various of the foregoing equations are susceptible of modificationin accordance with system requirements, for example, or in accordancewith individual limitations of system components. For example,operational characteristics of one or more stepper motors or servosimplemented in stage or optics component motion control may dictate thevalues for n_(z) or n_(y) in some instances. Additionally oralternatively, some circumstances may require or benefit from dynamiccomputation or adjustment of n_(z) and n_(y); one such circumstance mayinclude, for example, acquisition of image data at a plurality ofwavelengths during multi-spectral scans.

In that regard, motion control of various system components may beprovided or accommodated by electronics 150 as set forth above withreference to FIG. 1. Dynamic configuration of MAI scan motion parametersmay be readily implemented by one or more microprocessors ormicrocontrollers associated with, incorporated into, or operably coupledto electronics 150.

FIG. 11 is a simplified side view of an object to be imaged, and FIGS.12A-12C are simplified side view illustrations of an image planetraversing an object to be imaged using one digital multi-axisintegration method. The z and y directions are indicated by thecoordinate axes in FIG. 11. For illustrative purposes, the 3D object tobe imaged is represented by the 2D grid and designated by rows 1-4 inthe z dimension and columns a-d in the y dimension; it will beappreciated that the object to be imaged may extend in the x direction,i.e., normal to the plane of FIG. 11, as well. A smaller representationof the FIG. 11 object is reproduced in FIGS. 12A-12C; additionally,though the shapes are slightly different, it will nevertheless beunderstood that the object depicted in FIGS. 11 and 12A-12C mayrepresent object 199 illustrated in FIG. 1B.

Turning now to the sequence illustrated in FIGS. 12A-12C, a side view ofthe image plane is represented by the darkened grid squares, and thearrows indicate the direction of image plane motion relative to theobject to be imaged; as with the object depicted in the figures, it willbe appreciated that the image plane generally extends in the xdirection, i.e., normal to the plane of FIGS. 12A-12C. It is also notedthat the arrows depicted in the drawing figures are not vectors; in thatregard, the arrows are only intended to indicate the direction ofrelative motion for the image plane from one scan position to the next,and are not intended to convey information regarding the velocity ofsuch motion.

The shading in the representation of the object to be imaged is providedto indicate the number of times a particular location of the object hasbeen imaged by the image plane. The partially exploded view (i.e., theseparation between the object and the image plane) is provided forclarity.

For example, when at scan position 1 in FIG. 12A, only the right portionof the image plane in the figure may be acquiring image data; onlyportional of the object is within the image plane, and the arrowsindicate the motion of the image plane at this point in the scansequence. When at scan position 2 in FIG. 12A, the image plane hastraversed in the y direction as indicated (i.e., to the right) relativeto its location at scan position 1; in conjunction with the indicated ztranslation, the image plane at scan position 2 may be acquiring data atportions a2 and b2 of the object. The locations (on the object) of dataacquired at earlier or previous scanning positions in the sequence areindicated above the image plane in FIGS. 12A-12C. In accordance with theTDI embodiments set forth above, these data depicted above the imageplane may have been read from the imaging device (such as detector 141in FIG. 1A, for example) during the synchronous z and y motion.

It will be appreciated that at scan position 4, for example, and atother points during the scan sequence, the image plane may reversedirection; accordingly, z translation may be omitted at this point inthe sequence. As indicated in FIG. 12A, the image plane may remain atthe same z location for scanning operations which occur at scanpositions 4 and 5. While the image data acquired at position 5 will beat the same z location as those acquired at scan position 4, the imageplane has continued traversing in the y direction relative to its ylocation at scan position 4.

In the exemplary digital MAI scan of FIGS. 12A-12C, each z position istraversed by the image plane three times during the scan sequence; inconjunction with relative motion in the y direction, the image plane mayacquire image data from each location a1-d4 on the object twice. Forexample: image data are acquired for locational when the image plane isat scan positions 1 and 8; similarly, image data are acquired forlocation a2 when the image plane is at scan positions 2 and 7; and soforth.

In the foregoing embodiment, the equation n_(z)=3 n_(y)−1 may besatisfied where the number of z increments in the scan sequence equalseleven (n_(z)=11); that is, the twelve sequential scan positionsrepresent eleven incremental movements in the z direction, starting fromscan position 1.

As noted briefly above, integral multiples of n_(z) may be employed incertain embodiments effectively to increase the z scan frequency.Alternative computations may also be used to calculate n_(z) as afunction of n_(y), additional factors, or some combination thereof insome embodiments. As with the analog MAI scan embodiments describedabove, digital MAI scans techniques may employ a variety of alternativez scan frequencies depending upon system requirements, componentcapabilities, multi-wavelength scan strategies, and so forth. Motionparameters such as lateral velocity and z scan frequency may bedynamically adjusted or selectively altered, for example, by controlelectronics 150 or similar programmable electronic components. In someembodiments, for example, electronics 150 may selectively adjust motioncontrol parameters in accordance with feedback, data, or otherinstructions transmitted by image processor 142.

Upon examination of FIGS. 12A-12C, it will be apparent that image planedimensions, particularly when considered relative to the size of theobject to be imaged, may affect the total amount of data acquired duringan MAI scan, in general, and when the image plane is at specific scanpositions, in particular. In that regard, if the image plane depicted inFIGS. 12A-12C were larger in the y dimension than that shown, image datafor additional object locations may be acquired, for instance, when theimage plane is at scan positions 1-3 and 9-12.

FIG. 13 is an MAI summary. The FIG. 13 summary represents the image dataacquired during the digital MAI scan described above with reference toFIGS. 12A-12C. One or more data processing hardware or softwarecomponents, for example, integrated with image processor 142 orassociated with electronics 150 depicted in FIG. 1A may organize orotherwise manipulate image data acquired during the MAI scan.

Aspects of the present invention have been illustrated and described indetail with reference to particular embodiments by way of example only,and not by way of limitation. Those of skill in the art will appreciatethat various modifications to the exemplary embodiments are within thescope and contemplation of the present disclosure. It is intended,therefore, that the present invention be limited only by the scope ofthe appended claims.

1. A method for imaging biological specimens using an imaging chiphaving rows and columns, the method comprising: capturing a pixel imageof an object on the specimens; shifting the pixel image in the columnardirection of the imaging chip; moving the object in synchronous motionwith the pixel image; and reading out voltage values from the bottom rowof the imaging chip until a plurality of the
 2. A method for imaging asample using an imaging device, the method comprising: moving theposition of an image area on the sample along one dimension of thedevice; imaging a spot on the image area continuously until the imagedspot is moved out of the detection range of the device; and adjustingthe speed of the movement for adequate exposure time.